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ABSTRACT 


The major problem in the Department of Defense’s acquisition of software 
systems is the growing number of cost and schedule overruns that result in failed 
software acquisitions. Cost and schedule overruns are the consequence of the software 
development models selected, inaccurate estimation of size, time, and cost, the instability 
of user requirements, and poor decision-making by acquisition managers. Commercial 
practices of requirements definition, vendor selection, development process, business 
practices, integration, development, and testing, maintenance, and rights in data were 
compared with equivalent Department of Defense practices. Commercial solutions are 
the implementation of open source standards and architectures, iterative software 
developments, increased collaboration among competing vendors, and the incorporation 
of software reuse. The Department of Defense’s non-profit venture capital models utilize 
key practices, such as deal syndication and incremental funding, which are instrumental 
in managing risk and could be incorporated into how the DoD acquires software. 
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EXECUTIVE SUMMARY 


The Department of Defense and commercial industry are both motivated to 
acquire and develop the newest cutting-edge technologies. However, the funds necessary 
to develop innovative technologies are only appropriated under a certain set of 
expectations. Commercial industry is primarily concerned with taking a promising idea 
and molding it into a product line that will improve the company’s bottom line, by 
increasing revenues or profit margins. Meanwhile, the DoD is focused solely on 
attaining a product with 100% user-defined functionality at the lowest cost possible and 
in a timely manner. Compared to Commercial industry’s focus on achieving a significant 
level of user functionality, typically around 85%, and then increasing the functionality 
over a period of time, the DoD’s requirement to meet 100% of the user-defined 
functionality in a timely manner is the reason there are so many software acquisition 
failures. 

The Department of Defense has begun to explore alternative methods of 
procuring innovative technologies, most notably through the means of Venture Capital. 
In-Q-Tel, the Central Intelligence Agency’s venture capital arm, was the DoD’s first 
attempt at applying Venture Capital practices to improve the speed with which 
technologies are developed and deployed to the operational environment. In-Q-Tel 
invests in more risky projects/companies in comparison to the remainder of the 
government sector. In-Q-Tel funds companies involved in the research and development 
of cutting edge technology, which are inherently risky investments. However, with an 
increase in risk also comes the increase in reward. The rewards in this case are the speed 
with which commercially viable technologies can be identified, modified, and 
transitioned into the Department of Defense to enhance military capabilities and the 
added value of investing in commercial intellectual property early in the development 
lifecycle in comparison to when the technology is more mature and costly. 


xv 



A brief look at recent and past investments by the government shows a tendency 
towards large capital appropriations for projects. The government seeks out solutions to 
existing military problems and deficiencies by attempting to develop broad solutions that 
can be scaled to multiple platforms and areas of interest, but must meet 100% of user- 
defined requirements. These projects are often cancelled years later upon the realization 
that the product currently in development does not fill the initial specifications or solves 
only part of the problem for which it was intended. One of the possible solutions to this 
problem of a one-stop solution is the use of open source community and standard 
architectures combined with the goal of software modularization. If the government were 
to fund smaller, more manageable software projects they would increase their chances for 
success, while also spending less money. Additionally, through an analysis of 
commercial practices a set of recommendations was developed to improve the overall 
probability of successfully acquiring software systems within the Department of Defense. 
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I. INTRODUCTION 


A. BACKGROUND 

The Department of Defense is neither acquiring nor developing the newest 
innovative technologies at the same pace as the private sector. The Department of 
Defense’s Acquisition System hinders the speed with which technological advancements 
are made in the military. This lengthy acquisition process, a requisite for fielding new 
systems, is directly responsible for the slow transition of technology from the 
developmental laboratory to the operational environment as well as the lackluster rate of 
program success. 

The Department of Defense Acquisition System is intended to be a risk 
management tool designed to reduce the risk of project failures. However, the reduction 
in the number of project failures comes at the expense to the amount of time required to 
field a new system. “One of the major challenges confronting DoD is to achieve rapid 
(within 6-18 months) exploitation of innovative, commercial infonnation technology 
products in national security systems.” 1 Compared to the private industry’s short cycle 
times that can be measured in months, the Department of Defenses’ technology turnover 
time is measured in years. On average, it requires approximately ten years to transition 
from research and development to deployment. 2 Additionally, when the system is 
eventually fielded it is often almost two generations behind current technologies available 
in the marketplace. 3 

Congress has recognized the restrictive nature of the acquisition process and the 
inherent problems that result from its required usage. According to the House Report 
associated with the Defense Appropriations Act of 2003, “the Navy’s research and 
acquisition community historically has had great difficulty in transitioning innovative 

1 “Information and Command and Control.” Aerospace America (December 2004). 

2 Paul Mann, “Profit Incentives Urged for Defense R&D.” Aviation Week & Space Technology 

(March 2000). .~ .. 

3 Ibid. 
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technologies for government research organizations and the commercial marketplace to 
active deployment and procurement programs. Due to the constraints of internal 
planning and budgeting processes, and the stifling legacy of ‘programs of record’, new 
Navy systems are often fielded with a high degree of obsolescence...” 4 Therefore, if the 
Department of Defense is expected to keep up with speed of innovation evident in the 
commercial marketplace, it must begin to investigate alterations to its current Acquisition 
System. 

The Department of Defense’s acquisition of software systems over the last few 
decades has been a major cause for concern. Software system acquisitions typically fail 
as a result of one or more of the following factors: the inherent complexity of the 
software system, poor estimation of size, time, and cost, unstable user requirements, and 
poor decision-making by acquisition managers. “The development, acquisition, and 
delivery of software are essential to the Navy’s ability to successfully conduct its 
business and perform its missions.” 5 

Many of today’s cutting-edge weapon systems are increasingly dependent upon 
software. According to Lisa Pracchia, a member of Naval Air System’s Command 
Software Resource Center, “software enables a myriad of complex capabilities from 
massive data fusion across geographically disparate large-scale sensor systems, to 
decisional systems that automatically select the most appropriate weapon and platfonn to 
attack a given target.” 6 Examples of software-intensive systems include flight control 
systems, fire control systems, C4I systems, and autonomous mission planning systems; 
all of which would not be mission-effective without software. Within the past few years, 
software has even made great strides into improving the battlefield performance of 
infantry soldiers in various combat environments. These technological advancements are 
accompanied by an increase in software’s fraction of the weapons system’s life cycle 
cost. This increase in software costs is further magnified by optimistic cost estimates and 
delays in development and integration phase schedules. 

4 Department of Defense Appropriation Act, 2003, House Report 107-532 U.S.C. (2003). 

5 “Panel: Defense Software Problems Put Info Dominance ‘At Risk’.” Inside the Navy (July 2006). 

6 Lisa Pracchia, “Improving the DoD Software Acquisition Processes.” CrossTalk (2004). 
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Despite the Navy’s dependence upon reliable software systems, over the last 
decade an increasing number of software development initiatives have experienced major 
cost and schedule overruns. According to the Undersecretary of Defense, Jacques S. 
Gansler, systems currently in development are increasingly dependent upon software and 
it is from software issues that spawn many of the problems in cost and schedule impact 
arise. 7 A recent naval study conducted by the Naval Research Advisory Committee 
concluded that the Department of Defense must reconsider their current acquisition 
process for fielding new software systems. “The Navy has problems specifying, 
developing, acquiring, testing, maintaining, and researching software systems...The 
department also lacks experienced software acquisition professionals...Attempts to 
address testing, security, and interoperability are often too late... and the Navy is not 
investing enough money in software research.” 8 The software that must be integrated 
with newly developed and pre-existing weapon systems is driving the cost of building the 
Navy’s future fleet of warships even higher. 

Successful software acquisitions, as recognized by the Department of Defense, 
meet two specific criteria - 100% requirement compliance and the development of the 
software technology in a timely manner. The DoD recognizes a window of 5-8 years as 
an acceptable timeframe for development. Compared with commercial industry’s 
timeframe for development of 6-18 months, it is no surprise the DoD is lagging with 
respect to the development of innovative technologies. The commercial industry is able 
to measure their technology development timeline on a scale of months rather than years, 
as a result of meeting core user requirements through the incorporation of commercial- 
off-the-shelf (COTS) products and software reuse. Therefore, the commercial industry’s 
measure of success is meeting an acceptable level, generally 85%, of user requirements 
within 6-18 months. 


7 Paul Mann, “Profit Incentives Urged for Defense R&D.” Aviation Week & Space Technology 
(March 2000). 

8 “Panel: Defense Software Problems Put Info Dominance ‘At Risk’.” Inside the Navy (July 2006). 
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B. 


SCOPE OF THESIS 


This thesis provides an in-depth assessment on the current structure of Naval 
software system acquisition and whether or not it could benefit from the implementation 
of venture capital investment and management practices coupled with commercial 
software practices. Chapter II discusses the different software development models 
utilized in Department of Defense software acquisitions, software cost management, and 
the most common reasons for software acquisition failures in the DoD. Commercial 
practices for acquiring software are contrasted with the Department of Defense’s current 
practices in Chapter III. Chapter IV explores the Department of Defense’s 
implementation of non-profit Venture Capital firms. Conclusions and recommendations 
are discussed in Chapter V. 

C. THESIS STATEMENT 

The adoption of venture capital practices, such as deal syndication and 
incremental funding, coupled with the commercial software practices of open source 
standards and architectures, iterative software developments, and increased collaboration 
amongst competing vendors, will increase the number of successful software 
acquisitions. 
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II. WHY DEPARTMENT OF DEFENSE SOFTWARE 
ACQUISITIONS FAIL 


Within the Department of Defense, successful software projects are the exception. 
Most experience significant cost and schedule overruns leading them to exceed their 
budget and resulting in program termination. These acquisitions are capial-intensive and 
known for being significantly high-risk projects. The Standish Group’s annual Chaos 
Report tracks Information Technology successes and failures (See Figure 1). According 
to the figure below, it is evident that as the level of funding increases the probability of 
success decreases rapidly. 


Project Success 

Smaller initiatives fare better at reaching goals than larger projects do. 

More than $10 million 1 2% 


$6 million to $10 million 
S3 million to SGmillion 
$750,000 to $3 million 
Less than $750,000 

SOURCE: THE STANDISH GROUT 


11 % 


23% 


32% 


46% 


Figure 1. Software Project Success [From: Schwartz] 


In addition to significant capital appropriations, DoD software failure typically 
results from the software development model selected, unstable user requirements, and 
the program manager’s software cost management practices. 
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A. 


SOFTWARE DEVELOPMENT MODELS 


The most common software development models used in Department of Defense 
Software Acquisitions are the Waterfall and Spiral models. Each of these models is 
assessed on their ability to meet 100% user-defined requirements and the time required to 
deploy the technology. 

1. Waterfall Model 

The Waterfall model is a highly structured, documentation-intensive, sequential 
model, which replaced the initial “code and fix” method of software development and is 
now considered to be the traditional approach to software development. This model 
provides a well-documented system for increasing both the success of large software 
projects and the accuracy of cost and schedule estimates. However, its sequential nature 
prevents feedback between phases. Moreover, due to the fact that a working product is 
not available until late into software development, mistakes or errors in earlier phases 
often propagate through later phases. The Waterfall model is often successfully applied 
to low risk projects where software requirements are well understood and are not 
expected to change throughout the project life cycle. 9 

The Department of Defense favors the Waterfall model due to its rigid structure 
and the amount of documentation it requires. These characteristics are perceived by the 
DoD to be essential in minimizing program risk. However, user-defined requirements are 
constantly changing in DoD software projects and therefore present a significant 
challenge to software developments utilizing this model. Since there is no method of 
addressing changes in user requirements throughout the software development process, 
the resulting product often does not meet the required level of 100% compliance. 
Additionally, a reduction in funding or program termination would leave little to be 


9 “Guidelines for Successful Acquisition and Management of Software-Intensive Systems: Weapon 
Systems, Command and Control Systems, and Management Information Systems.'” Department of the Air 
Force: Software Technology Support Center (February 2003). 
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salvaged for future work, since the Waterfall model does not produce a working product 
until late in the development process. Therefore, although this model is perceived to 
reduce risk, in reality it actually increases risk for the Department of Defense. 

2. Spiral Model 

The Spiral model’s main purpose is to reduce risk by identifying high-risk 
problems as they arise throughout the software life cycle. With each iteration of the 
model, objectives, alternatives, and constraints are identified, risk analysis is conducted, 
the software product is further developed, and plans are reviewed for the next cycle. The 
model is adept at incorporating new technologies, reducing risk, and developing a system 
that is far more responsive to user needs. However, this model usually increases 
development cost and is more difficult to manage than others. The Spiral model should 
be considered for projects with high risk and an increased likelihood of evolving 
requirements. 10 

The Department of Defense utilizes the Spiral model in its software developments 
since it provides more than adequate analysis of risk throughout the development process. 
This is accomplished through the significant amount of documentation that is generated 
with each iteration of the model. The resources required to persistently document the 
software development, analyze risks, and identify objectives and alternatives directly 
increase the costs of developing the software. Additionally, completing these tasks 
consumes a substantial portion of valuable development time, which could result in 
compounded schedule overruns that delay the system’s time to deployment. Therefore, 
the Spiral model is not the ideal software development model to acquire software for the 
Department of Defense. 


10 “Guidelines for Successful Acquisition and Management of Software-Intensive Systems: Weapon 
Systems, Command and Control Systems, and Management Information Systems.” Department of the Air 
Force: Software Technology Support Center (February 2003). 
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B. UNSTABLE USER REQUIREMENTS 


The notion that requirements will change over time is well understood, but the 
practice of developing software systems with a model and architecture that support 
changing requirements remains ignored. Another factor that must be considered is how 
and when to incorporate new or modified requirements. If requirements are revised to 
frequently, cost and schedule overruns are likely since the software development is not 
able to gain much traction. Therefore, it is imperative to control the modification of 
requirements to minimize requirements creep and increase the chances of developing a 
software system that meets the user’s required level of functionality. One way to ensure 
requirement compliance and increase the likelihood of meeting required functionality is 
to solicit feedback from the software system’s user. User involvement is critical to 
determine whether perceived user needs have been correctly translated into software 
functionality. 

C. PROGRAM MANAGER SOFTWARE COST MANAGEMENT 

Project managers are tasked with controlling cost fluctuations, while 
simultaneously achieving performance milestones and scheduled deliverables. In 
addition, the budget from which they receive funding is based upon their own estimates 
of software development. With respect to software development projects, the majority of 
cost pertains to staffing, which is a numerical figure determined by the size of the 
software system being developed. In order to accurately estimate this cost, one must 
have knowledge of the number of people needed and the amount of time necessary to 
complete their assigned task. The accuracy of these estimates is often over-estimated as 
they are simply the product of guesswork formulated from an analysis of an analogous 
system’s historical data. Program manager estimation of cost, time, and size and method 
of determining software development cost contribute to the failure of DoD software 
acquisitions. 
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1. Inadequate Estimation of Cost, Time, and Size 


An inability to adequately estimate cost, time, and size is the fundamental reason 
for software-intensive development cost and schedule overruns. The effects of these 
optimistic estimates are compounded in software acquisitions since predicting size and 
complexity of software developments is challenging when there is no analogous system 
to gauge the accuracy of a program manager’s estimates. 

The increased optimism of contractor and program manager estimates stem from 
Congress’ spending and budgeting scrutiny. Since best-case estimates increase a 
program’s chances of being funded or approved, there is little motivation for acquisition 
professionals to provide more pessimistic estimates that account for deviations in the 
software development process and incorporate worst-case scenarios. Additionally, 
contractors are motivated to egregiously underestimate a software program’s cost since 
the DoD Acquisition System generally awards funding to the contractor with the lowest 
cost and schedule estimates, given they meet all other requirements. Once awarded the 
contract, it is not uncommon for there to be increases in cost and schedule throughout the 
program’s lifecycle that more accurately reflect the program’s true cost and delivery 
timeline. 

These optimistic estimates often do not account for the cost of a chaotic software 
development. Programs tend to get in trouble in small, progressively compounding 
increments. When the software system is expected to experience cost and/or schedule 
overruns due to inadequate estimation, it is not uncommon for the developer to reduce the 
number of requirements to be fulfilled and the amount of testing to be conducted. 
However, the elimination of user requirements and reduction in the number of tests to be 
run significantly impact the final product. In these cases, the software system falls short 
of meeting the user’s required level of functionality and/or is riddled with defects, which 
limit its deployability to the operational field. Therefore, the inadequate estimation of 
time, size, and cost results in unrealistic delivery schedules and insufficient program 
funding. 


9 



2 . 


Method for Determining Software Development Cost 


The key measurement for detennining software development cost is the size of 
the software system being developed. The predominant method for estimating software 
size is through source lines of code (SLOC). The SLOC approach is most commonly 
practiced despite three significant drawbacks. First, the lack of a universal standard for 
what determines a source line of code makes it difficult to compare multiple developer 
estimates. Second, since the developer is compensated based on a flat rate per source line 
of code he has an incentive to increase the number of lines of code required for the 
software, directly increasing compensation. This effectively removes efficient 
programming practices as the developer is motivated by compensation and not efficiency. 
Lastly, in most software development programs, multiple programming languages are 
used and estimates of cross-language lines of code are inconsistent, exponentially 
increasing the number of potential pitfalls for developing software. 

Lastly, initial releases of the software development often perform inadequately 
and the cost of modification is prohibitive. If problems are not caught in a timely 
manner, the cost of making and correcting defects often soars beyond acceptable levels. 
Unfortunately, this results from scheduled milestones that force the developer to choose 
between delivering the product as scheduled with known deficiencies or delaying its 
release and facing the consequences of schedule delays. In addition, there always exists 
the possibility of unknown hardware deficiencies that may require costly software 
modifications. 

Schedule overruns, no matter how small and insignificant they may appear, 
almost always lead to an increase in costs. As a result of the numerous challenges that 
result from the poor estimation of software systems and inadequate software development 
models, it is imperative that the Department of Defense takes steps to improve software 
acquisitions. 
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III. SOFTWARE ACQUISITION: A COMPARISON OF 
DEPARTMENT OF DEFENSE AND COMMERCIAL 

PRACTICES 


A. BACKGROUND 

The Department of Defense’s software acquisition practices can be compared 
with common commercial practices of acquiring software. The following comparison is 
conducted on the basis of two key issues — (1) specific industry methods and (2) the 
integration of commercial-off-the-shelf (COTS) products. Only the specific industry 
methods used in acquiring software systems are similar to those currently developed by 
the Department of Defense and the practices of incorporating COTS products into 
complex software systems are compared. An analysis of COTS products is important 
since research and development has been increasingly coming from privately funded 
projects as opposed to the traditional sources - government and academic laboratories. 
Therefore, the ability to incorporate commercial tools has become more valuable and the 
ability to leverage private sector projects means faster implementation of technology - 
rather than awaiting custom-designed tools. 11 An in-depth analysis of these issues 
exposes both the benefits and risks associated with implementing certain commercial 
practices within Department of Defense software acquisition programs. 

B. COMMERCIAL INDUSTRY VS. DEPARTMENT OF DEFENSE 

SOFTWARE SYSTEMS 12 

The software systems acquired and maintained by both the commercial industry 
and the Department of Defense can be classified into three major groups - Real-Time 
Embedded Control systems, Information systems, and Command, Control, 
Communications, Computing, and Intelligence (C4I) systems. Characteristics of Real- 


11 Gary Alien, “For Tech’s Sake: Can In-Q-Tel’s investment model translate into other agencies?” 
Washington Technology (22 April 2004). 20 May 2007 http://www.washingtontechnology.com . 

Michael E. DeRiso and Jack R. Ferguson, “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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Time Embedded Control systems include interrupt-driven processes, large numerical 
processing requirements, small databases, tight real-time constraints, and well-defined, 
diverse user interfaces. In addition, their requirements and design are driven primarily by 
performance constraints. Examples of these software systems include aircraft control 
systems and fire control systems. Information systems are transaction based, consisting 
of moderate numerical processing requirements and large databases. In addition, these 
systems have flexible time constraints and user interfaces. The requirements and design 
of Information systems are driven by their user interface. Examples include accounting, 
personnel, and supply management systems. Lastly, C4I systems require large numerical 
processing requirements and large databases. They also have near-real time requirements 
and user interfaces that vary in complexity and flexibility. Both performance and user 
interface drive the requirements and design of these types of software systems. An 
example of a C4I system is a missile warning and control system. Software systems that 
fall within the first two categories are commonly developed by both the Department of 
Defense and commercial industry. However, the third type of software system, C4I 
systems, applies mainly to the Department of Defense. In addition to these three major 
types of software systems, commercial industry also develops general-purpose software 
such as operating systems and task oriented solutions to enhance office productivity. 

The following sections and tables contrast common commercial practices with 
those typically used in Department of Defense software programs, focusing on the areas 
of requirements definition, vendor selection, development process, business practices, 
integration, testing, delivery, maintenance, and rights in data. 

1. Requirements Definition 

Commercial industry is able to better define requirements due to the nature of the 
relationship between the buyer and the user. In commercial industry the buyer and user 
are typically the same, while in DoD acquisitions they are rarely the same. In the DoD, 
the source of the problem is the program office, which does a poor job of translating 
mission needs into product requirements. 
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During the process of defining requirements, commercial industry has adopted the 
practice of researching existing product solutions and incorporating their technology or 
increasing their functionality to meet their specific needs. Thus, it is evident that 
commercial industry is more willing than the Department of Defense to trade 
functionality for the possibility to decrease cost and schedule. This increased flexibility 
enables commercial industry to develop software systems with the focus of delivering a 
functional product ahead of schedule and then evolving it to include later requirements. 
In addition, one of the best commercial practices is the formation of an Integrated 
Product Development (IPD) team, which consists of all parties to be involved throughout 
the program’s life cycle. With this team in place, all parties are able to effectively 
communicate any rising concerns throughout all phases of software development. 

Also, it is common practice for commercial software companies to think in terms 
of product lines that fit into standard architectures. This practice enables them to achieve 
the goal of creating software products that can easily be tailored to meet different user 
requirements. 

An analogous practice to thinking in tenns of product lines for the Department of 
Defense would be to focus on building tailored software modules that can be plugged into 
a standard open architecture. Additionally, the Department of Defense could also 
improve by requiring acquisition professionals to set up test acquisitions, which would 
enable them to explore various functionality/cost tradeoffs. 13 Through the 
implementation of Advanced Concept/Technology Demonstrations (ACTD), the 
Department of Defense has begun to explore the advantages and disadvantages of 
systems developed by competing contractors. 

Lastly, the adoption of a team approach could potentially increase the 
government’s visibility into the supplier’s present, past, and future programs thereby 
decreasing program risk. Most importantly, the addition of the software system’s end 


13 Michael E. DeRiso and Jack R. Ferguson, “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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users to the IPD team is vital to the success of any software system acquisition. Table 1 
contrasts commercial industry practice and the Department of Defense practice pertaining 
to requirements definition. 


Requirements Definition 

Commercial Practice 

DoD Practice 

Requirements based on strategic plan and 
market analysis. 

Requirements based on requesting 
command’s Mission Need Statement. 

Requirements based on life-cycle resource 
constraints. 

Requirements based on annual budget 
resource constraints. 

Detailed requirements generated by team 
consisting of users, subject matter experts, 
and system engineers. 

Detailed requirements generated through 
collaboration of the buyer with the user. 
Team consists of subject matter experts and 
acquisition personnel. 

Functional specification is modified by 
knowledge of existing product solutions. 

Functional specification is created with 
little to no regard for existing product 
solutions. 

Vendors involved early in study, analysis, 
prototyping. Emphasis on reuse and 
evolution of existing systems. 

Contract may include prototypes, but 
contractor involvement in pre-award 
discussions is discouraged. 

Additional requirements tradeoff decisions 
concerning complexity and schedule for 
reduced time to field. 

Limited flexibility to trade off requirements 
creep versus complexity and schedule. 

Summary: 

Commercial practices are more flexible and open between users and suppliers. In the 
commercial sector, there is more willingness to adjust requirements based on availability 
of products and thus to field a system earlier and evolve it to include more capability. 


Table 1. Requirements Definition. [From: DeRiso and Ferguson] 


2. Vendor Selection 

The differences in vendor selection between commercial industry and Department 
of Defense are significant. From Table 2, it is evident that the commercial industry has 
an entirely different relationship with suppliers. These differences include which vendors 
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each solicit and how each vendor is selected. For instance, in commercial industry 
vendors are encouraged to form teams with other qualified vendors to create an 
environment that fosters rapid technology development through tradeoffs in cost and 
schedule. In addition, the formation of teams filled with qualified vendors all working 
cohesively towards the development of the software system provides the foundation for 
future collaboration on new and innovative software ventures. 

Commercial industry’s objective in software development is to obtain the best 
possible product at a reasonable price and as soon as possible. When comparing this to 
the Department of Defense’s objective of obtaining a product with 100% user- 
requirement compliance at the lowest possible price, it is not surprising that a significant 
number of software acquisition programs often experience cost and schedule increases, 
ultimately leading to program termination. However, their ability to negotiate with 
suppliers and to engage in long-tenn contractual relationships covering both the 
development and maintenance phases, similar to commercial industry, is not pennitted 
under current regulations. In addition, the Department of Defense’s requirements to 
maintain public trust and to be rigorously fair constrain its ability to modify existing 
vendor selection techniques. 14 


14 Michael E. DeRiso and Jack R. Ferguson, “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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Vendor Selection 


Commercial Practice 

DoD Practice 

Solicit multiple qualified vendors. 

Encourage teaming with a view to attaining 
a relationship that covers the entire life 
cycle and fosters tradeoffs in cost and 
schedule. 

Solicit all possible vendors. Vendor 
proposals must meet 100% of 
requirements. Teaming seldom encouraged 
since development and maintenance are 
usually separate entities. 

Compare vendor history and experience. 
Maintain long-term relationships for future 
business. 

Can compare previous performance, but 
long-tenn relationships generally 
discouraged. 

The organization that will be responsible 
for a system over its full life cycle is 
heavily involved from the beginning. 

Maintenance organization not usually 
involved in vendor selection process. 

Overall goals: Obtain product at reasonable 
price and as soon as possible. 

Overall goal: Obtain the lowest cost 
product that complies with all 
requirements, but equal opportunity is a 
must. 

Few review and approval steps once 
vendor has been selected. 

Review and approval process more 
structured and complex once vendor 
selected. 

Past performance weighed heavily in 
selection process and is often primary 
factor. 

Past performance considered, but typically 
only a minor fact. 

Summary: 

Commercial practices allow for more flexibility in vendor selection, while DoD vendor 
selection is forced by use of pre-defined metrics for proposal evaluation. The constrained 
process of vendor selection for the DoD is a result of their requirement to maintain the 
public trust. Commercial practices encourage vendors to offer best solution, despite the 
fact that it may not meet 100% of the requirements. In addition, under these practices, 
teaming and long-term relationships are more easily accommodated and are directly 
attributable to the success of the software development. 


Table 2. Vendor Selection. [From: DeRiso and Ferguson] 
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3. Development Process 


Commercial industry and the Department of Defense share traditional approaches 
to the development process, however a few key differences increase the potential for 
success in commercial industry. Both commercial industry and the Department of 
Defense have implemented reviews to monitor the level of risk associated with programs 
throughout all phases of software development and to ensure that adequate progress is 
achieved against major milestones. Yet, the key differences include the incorporation of 
commercial-off-the-shelf products, considerations for product architecture, the level of 
user involvement, and the application of prototyping. 

The enhancement of existing product solutions to achieve desired user 
requirements is a well-known process of commercial industry. The main reason for the 
consistent success of this practice is their consideration for product architecture 
throughout the development process. Commercial product solutions often fit into 
standard architectures, which enable them to take advantage of software reuse. 
Unfortunately, the practice of software reuse is often ignored by the Department of 
Defense in their software acquisition programs as a result of their desire for meeting 
100% of the user requirements. As a result, the possibility that development funding is 
wasted by multiple efforts is high. 15 The commercial industry successfully navigates this 
pitfall by translating buyer requirements to more “general purpose” requirements, which 
fulfill the unique needs of the user, but allow for future modifications. 16 This practice is 
evidenced by their focused development of modules within their software product 
solutions. Modularity enables the commercial vendor to meet multiple user needs by 
compartmentalizing unique user requirements to fit within standard industry 
architectures. Therefore, the Department of Defense should consider utilizing standard 
architectures, which would enable multiple vendors to contribute to a software project, 


15 Joab Jackson, “Report advocates open-source approach for software acquisition.” Government 
Computer News (20 July 2006). 

16 Michael E. DeRiso and Jack R. Ferguson, “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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with fewer inter-operability issues. In addition, this would also enable them to 
incorporate existing commercial software products into software projects to fulfill user 
needs, thus reducing program cost and required time to product deployment. 

The difference between the levels of user involvement within commercial 
industry in comparison to that in Department of Defense software programs is significant. 
This difference can be attributed to the role the user takes on in each domain. It is 
standard commercial practice for the user to be heavily involved throughout all phases of 
software development. Generally, this is accomplished through user representation 
within the Integrated Product Development team. This enables the user to communicate 
essential input concerning the design and development of the software system. These are 
key aspects that users involved with Department of Defense projects rarely have the 
opportunity to take advantage of. With respect to Department of Defense software 
projects it is common practice to limit the role of the user throughout software 
development. User input is solicited during the drafting of requirements and during the 
testing phases. Therefore, it is not unusual for certain aspects of the software system’s 
design and development to be unintentionally restrictive. 

Software prototyping is the process of creating an incomplete model of the future 
software system with limited functionality. Prototyping allows the users to evaluate the 
limited features of the software system to determine requirements compliance and to 
generate feedback that may not have been considered during the initial design and 
development phases. This process has several advantages. First, the software design and 
development team can obtain feedback from the users early in the project. Second, the 
buyer and the contractor can compare if the software prototype meets the software 
specification. Lastly, prototyping yields insight into the accuracy of initial cost and 
schedule estimates and whether the proposed milestones can be successfully met. Table 
3 provides a summary of common development process practices in both commercial 
industry and the Department of Defense. 
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Development Process 


Commercial Practice 

DoD Practice 

Vendor often tails existing systems and 
uses COTS. System designed to fit in a 
defined architecture or product line. 

Varies with application. Some systems use 
COTS. However, usually a new system that 
doesn’t reuse legacy software. Unique 
systems are built with little regard for 
architecture. 

Buyer may have heavy involvement in 
design and development as part of the IPD 
team. 

Formal, structured spiral, or waterfall 
model. Buyer oversees, but team approach 
is not typically emphasized. 

Informal reviews conducted to detennine 
progress against goals. 

Very formal reviews, which include 
technical details in addition to progress 
metrics. 

Heavy user involvement. 

Limited user involvement, but heavy buyer 
involvement. 

Vendor embraces one or more industry 
standards, which improves interface and 
integration with COTS products. 

Limited practice of integrating industry 
standards with government standards due 
to compatibility and security issues. 
Additionally, COTS products do not 
necessarily enhance government standards. 

Buyer requirements may be translated to 
more “general purpose” requirements for 
potential software reuse. 

Software systems are tailored to meet user 
requirements with little focus on designing 
reusable code. 

Management reviews and degree of 
oversight are commensurate with size and 
risk of program. 

Notably more detailed reviews and 
oversight performed. 

Prototyping common, with joint 
applications development teams working to 
clarify requirements and incorporate new 
requirements that do not affect cost or 
schedule. 

Prototyping seldom used, but becoming 
more popular. 

Summary: 

Commercial practices more flexible with likelihood of a team approach and is based 
toward reuse and modification of existing systems. Software system architecture is 
designed and developed with the future anticipation of product improvements. 


Table 3. Development Process. [From: DeRiso and Ferguson] 
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4. 


Business Practice 


There are many reasons for the differences in the way business is conducted in 
commercial industry in comparison to that of the Department of Defense. First and 
foremost, contracts within commercial industry are often informal, falling under the 
classifications of joint venture or partnership. Joint ventures and partnerships are 
characterized by the sharing of a program’s risk, which results from vested interests and 
the possibility for mutual economic benefit. Department of Defense contracts are 
significantly more formal and rigorously structured, but most importantly they lack the 
inherent motivation to successfully reduce cost. This lack of motivation for the 
contractor to reduce cost often stems from the level of vested interest in the project. In 
comparison to commercial industry, in which contracts are often entered into due to the 
project’s mutual economic benefit, the Department of Defense cannot begin to expect a 
reduction in program costs until they align the contractor’s interests with their own. 

Another major difference in business practices between these two sectors has to 
do with the nature of project funding. Commercial industry projects receive predictable 
annual funding, while Department of Defense projects often receive varying annual 
amounts of funding. Therefore, the unpredictability of Department of Defense funding 
could possibly decrease a contractor’s level of motivation for reducing costs. 

Lastly, there exists a difference in staffing policies that could also affect the 
success of the program. Commercial industry has the ability to hire\fire vendors and 
managers throughout the project’s lifecycle. Often commercial project managers have 
had significant experience in overseeing multiple programs throughout their 
development. This expertise could play a role in decreasing the level of risk associated 
with commercial projects as a result of their ability to recognize situations in which cost 
and schedule overruns may arise. Yet, Department of Defense project managers are 
detennined mostly by rotations of assignment, followed by qualifications, and finally 
training. Therefore, it is not uncommon for experienced program managers to be 
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replaced by less experienced individuals who are coming straight from training programs. 
Table 4 provides a brief summary of the differences in business practice with respect to 
commercial industry and the Department of Defense. 


Business Practice 

Commercial Practice 

DoD Practice 

Informal contracting, joint ventures, 
partnerships with mutual economic benefit 
and vested interest in success. 

Formal contract with little motivation to 
reduce cost. 

Oversight built on established 
relationships. 

Burdensome cost accounting procedures 
required; extensive oversight, reporting, 
and documentation requirements. 

Ability to hire and fire vendors and 
managers. 

Government personnel regulations, 
policies, and practices determine 
qualifications of its managers, rotations of 
assignment, and training. 

Predictable annual funding. 

Unpredictable annual funding. 

Summary: 

Commercial practice more flexible with greater incentives. 


Table 4. Business Practice [From: DeRiso and Ferguson] 


5. Integration, Testing, and Delivery 

Commercial industry and the Department of Defense differ in the types of testing 
each conducts. The Department of Defense conducts both operational testing and 
development testing per statutory mandate, while commercial industry typically conducts 
only development testing. 17 A separate authority conducts the operational testing in 
order to achieve an impartial, independent assessment of the software system. This 
testing is necessary because the software system must withstand the operational 
environment in which it will be used, e.g. the battlefield. Since the Department of 

17 Michael E. DeRiso and Jack R. Ferguson, “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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Defense is required to run both development and operational tests, the software project’s 
schedule and complexity is naturally increased. Another major difference revolves 
around the amount of beta testing conducted. Beta testing consists of releasing different 
versions of the software system to outside sources so that further testing can minimize the 
number of errors or bugs within the software. This practice enables the software 
development team to increase their product’s exposure to an increased number of future 
users. It is common practice in the commercial sector to release multiple beta versions 
for testing by other users, to ensure that the final software product has few faults. 
However, in the Department of Defense it is much less common for a software project to 
go through intensive beta testing. This lack of beta testing could also be responsible for 
the propagation of errors throughout the software development lifecycle, thus affecting 
the timetable for product delivery and increasing costs resulting from software rework. It 
is estimated that in fiscal 2006, the Department of Defense spent as much as $12 billion, 
or roughly 30 percent of its estimated budget of $40 billion for research, development, 
testing, and evaluation, on reworking software. 18 To put this number into perspective, 
large commercial companies spend just a small percent of its budget on rework. Thus, it 
is evident that there are undeniable problems associated with the methods in which the 
integration, testing, and delivery of software systems are conducted in the DoD. Table 5 
below compares common practices regarding integration, testing, and delivery of 
software systems within commercial industry and the Department of Defense. 


18 Patience Wait, “Weapons projects misfire on software.” Government Computer News (3 July 2006). 
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Integration, Testing, and Delivery 


Commercial Practice 

DoD Practice 

There usually exist major “cut-over” 
issues. 

Similar “cut-over” or transition issues. 

Sometimes difficult to assemble complete 
system in laboratory environment due to 
cost. Testing usually done in client’s 
facility. 

Usually integrate system in laboratory prior 
to operational testing. Development testing 
vs. operational testing required via 
statutory mandate. 

Beta testing widely used to quickly find 
errors. 

Minimal beta testing. 

Ultimate acceptance authority rests with 
buyer, not a separate organization. 

Structured, specified operational testing 
conducted by separate authority. 

Acceptance authority rests with buyer. 

Summary: 

Integration and functional testing seem appropriate to the need. DoD use of separate test 
agency adds time and complexity. 


Table 5. Integration, Testing, and Delivery. [From: DeRiso and Ferguson] 


6. Maintenance 

Software maintenance has been restructured in both the commercial industry and 
the Department of Defense. Historically, it had been commercial practice for the buyer to 
support the software system. However, it has become increasingly common to find the 
burden of software support being outsourced to independently contracted companies or 
supported internally by the vendor responsible for the software’s development. 19 
Commercial industry’s transition from organic support to vendor-supported maintenance 
is motivated by cost efficiency. It costs the software buyer less to require the vendor to 
maintain the software it developed then it would be to actually provide the software 
maintenance internally. In contrast, the Department of Defense provides software 
support through the use of depot maintenance as a result of their reluctance to be 

19 Michael E. DeRiso and Jack R. Ferguson. “Software Acquisition: A Comparison of DoD and 
Commercial Practices.” (1994). 
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dependent upon the vendor. “Depot maintenance is the act of repair, overhaul, upgrade 
or rebuild of weapons systems, support equipment, component parts, and embedded 
operating software programs when the level of effort required to meet specified 
conditions exceeds the capabilities of lower level maintenance activities (i.e. Direct 
Support and General Support).” 20 One of the primary reasons for depot maintenance is 
the increased level of responsiveness required to support software systems in the 
operational environment. Therefore, it is not practical to port commercial practices 
regarding software maintenance to the Department of Defense depot maintenance model. 
Table 6 provides a comparison of commercial industry and the Department of Defense 
maintenance practices. 


Maintenance 

Commercial Practice 

DoD Practice 

Organic support shifting to outsourcing or 
vendor. 

Organic support, with reluctance to be 
dependent on vendor. Use of depot 
maintenance makes interoperability issues 
more manageable. Must be responsive to 
user for critical systems. 

Summary: 

The DoD and industry have different requirements and must be careful when selecting a 
maintenance strategy appropriate to their needs. 


Table 6. Maintenance. [From: DeRiso and Ferguson] 


7. Rights in Data 

The Department of Defense generally demands all rights in data since they 
support and maintain the software system internally. In commercial industry, there are 
differing practices for who acquires the rights in data. In any custom development, the 
buyer typically gets all rights, but it is generally agreed upon in the contract that the 
vendor might retain the right to subsequent sales. However, if the buyer is purchasing a 

20 “Depot Maintenance Program Info,” US Army Materiel Command , U.S. Army, 10 May 2007 
http://www.amc.army.mil/G3/maior programs/dmdesc.htm . 
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tailored version of a standard software system, the buyer will only acquire rights to the 
tailored functionality, but not the standard system. This practice is common within the 
Department of Defense, but there exist numerous exceptions for proprietary material. 

The Department of Defense has experienced many problems deriving from the 
practice of acquiring rights in data. Generally, the government seeks rights in data in all 
software projects, to prevent vendor-lockup, which prohibits the Department of Defense 
from awarding alternative vendors projects to improve or increase functionality, 
pertaining to proprietary material. Due to the inherent proprietary issues concerning 
software reuse, especially in legacy systems, the Department of Defense began exploring 
the advantages/disadvantages of open source. Within the last year the Open Technology 
Development (OTD) Road Map report investigated various technology development 
methodologies. Since OTD’s inception, it has been advocating the adoption of open 
source standard architectures through sharing software code developed by the DoD, its 
affiliated contractors, and the open source community to increase the speed of the 
software procurement cycle. The report stated, 

Currently, within DoD acquisitions programs, software code is reused on a 
limited basis. For example, within an individual DoD program office, 
software code from a previous contractor may be shared with a new 
contractor taking their place. 21 

By opening up standard architectures, they can minimize the number of problems 
that arise as a result of proprietary material in future software programs. Table 7 
contrasts commercial industry practice and the Department of Defense practice pertaining 
to rights in data.” 


21 Joab Jackson, “Report advocates open-source approach for software acquisition.” Government 
Computer News (20 July 2006). 
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Rights in Data 


Commercial Practice 

DoD Practice 

If custom development, buyer gets all 
rights, but vendor may retain right to 
subsequent sales. 

Specified by contract. Government usually 
demands all rights for government use due 
to organic support, maintenance needs, and 
competition. 

If tailored version of standard system, 
buyer only gets rights to tailored parts. 

Similar to commercial, but many 
exceptions for proprietary material. 

Summary: Similar, but commercial is a little more flexible especially regarding resales. 


Table 7. Rights in Data. [From: DeRiso and Ferguson] 
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IV. DOD VENTURE CAPITAL MODELS 


The Department of Defense’s ability to identify, develop, and deploy innovative 
commercial technologies is limited. There are commercial technologies not yet identified 
by the DoD as a result of their limited visibility into the newest private sector 
innovations. The concept of funding a portion of technology development through 
venture capital “exploits venture capital’s efficiency in developing technology, its access 
to the growing commercial technology sector, its capacity to respond with agility to 
changing technology, and its ability to leverage additional resources throughout the 
development cycle.” 22 The firm’s required strategic focus is representative of the venture 
capital arms of a subset of today’s most distinguished public companies. For example, 
both Intel and Xerox have successfully leveraged their corporate expertise and industry 
knowledge to expand their company’s capabilities through strategic acquisitions and 
investments. Therefore, the Department of Defense began to explore the benefits of 
venture capital through a few government sponsored venture initiatives. 

A. GOVERNMENT SPONSORED VENTURE INITIATIVES 

There are currently a number of government sponsored venture initiatives within 
the Department of Defense. These initiatives span the entire spectrum of venture capital 
models, which ultimately seek to enable the identification and transference of technology 
from the commercial sector to the federal government. Government sponsored venture 
initiatives fall into two categories, programs that provide resources for commercial 
technologies showing promise in addressing specific militaristic problems and programs 
that serve as self-sustaining independent entities, setup to model a Venture Capital (VC) 
firm. The U.S. Navy Commercial Technology Transition Office (CTTO) serves as a 
bridge between the commercial sector and the military, matching commercial solutions 


22 Bruce Held and Ike Chang. “Using Venture Capital to Improve Army Research and Development.” 
Rand Corporation (2000). 
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with militaristic problems. However, the Central Intelligence Agency’s In-Q-Tel and the 
U.S. Army’s OnPoint Technologies are two examples of the Department of Defense’s 
most recent forays into government venture capital models. 

1. Commercial Technology Transition Office (CTTO) 

The Commercial Technology Transition Office is the U.S. Navy’s program 
established to leverage the unique expertise of the VC community in their ability to 
identify viable commercial technologies. It was created in 1999 by the Assistant 
Secretary of the Navy to improve the successful identification and rapid transference of 
technology to the U.S. Navy. The CTTO is involved throughout all aspects of 
technology insertion, including initial assessment of the basic technology, developing 
program milestones, funding, schedules, testing, and finally through acceptance by the 
Naval buyer. 23 Since 1999, the CTTO has successfully funded over 55 technology 
transition deals, each of which has been instrumental in either improving Naval 
capabilities or reducing total lifecycle costs for current systems. 

a. Mission 

The primary mission of the Commercial Technology Transition Office is 
the rapid transfer of technology to improve the U.S. Navy’s current capabilities. The 
CTTO serves as a matchmaker between Naval Acquisition Program Officers and various 
technology providers. In order to identify technologies that are assured to solve existing 
problems in the Department of Navy, the CTTO utilizes its Rapid Technology Transition 
Proposal Form to solicit input from Program Executive Offices (PEOs), System 
Commands (SYSCOMS), and Office of Naval Research (ONR) Program Managers. 
“Proposed technology transitions are often based on one or more factors such as program 
risk, schedule acceleration, improving system performance, and reducing acquisition 


23 “Commercial Technology Transition Officer (CTTO) - Deals,” Office of Naval Research, 
Department of the Navy, 12 May 2007 http://www.onr.naw.mil/ctto/deals.asp . 
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cost.” 24 Sources of technology include commercial industry, academia, and government 
sources, both Department of Defense affiliated and non-DoD affiliated. Figure 5 below 
provides an overview of the CTTO deal-making process. 



Figure 2. CTTO Deal Making Process Overview. [From: Commercial Technology 

Transition Office] 


b. Investment Requirements 

The difference between this program and other government sponsored 
initiatives lies with the CTTO’s requirements for investment in technology. The 
threshold for technological funding is much higher within the CTTO. Additionally, 

- 4 “Commercial Technology Transition Officer (CTTO) - Deals,” Office of Naval Research, 
Department of the Navy, 12 May 2007 http://www.onr.naw.mil/ctto/deals.asp . 
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potential technologies are evaluated on the basis of their Technology Readiness Level 
(TRL), which assesses the maturity of an existing technology by indicating the present 
status of development with respect to its lifecycle timeline. In order to minimize 
technology and program risks, the minimum Technology Readiness Level for investment 
is TRL 6, which indicates that a system prototype has been demonstrated in a relevant 
environment. The table below illustrates the different TRLs associated with potential 
technologies. 
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9 Actual system ‘flight proven* through successful mission operations (OT&E) 

8 Actual system completed and flight qualified' through test and demonstration 
(ground iflight) (DT&E) 

7. Systems prototype demonstration in a flight/space environment (System 
Prototype Test in Operational Environment) 

6 SystenVsubsystem model or prototype demonstration m a relevant environmen 
(Prototype Test in Relevant Environment) 


5. Component and/or breadboard validation in a relevant environment 

(Breadboard Integration) 

4. Component and/or breadboard validation in laboratory 
environment (Breadboard Integration) 

3. Analytical and experiment critical function and/or characteristic 
proof of concept (Component Development) 


2. Technology concept and/or application formulated (Invention) 
1. Basic principle observed/reported (Paper Study) 


Table 8. Technology Readiness Levels. [From: Commercial Technology Transition 

Office] 


Since most technologies funded by the CTTO have TRLs of 6 or 7, additional funding is 
often a requirement in order to yield a product that can be successfully deployed to the 
operational environment. Additional funding for technologies is secured through a 


30 




























Memorandum of Agreement (MOA), which outlines specific actions and resources 
necessary to complete the technology transition. Technologies selected for transition 
typically receive between $500,000 and $2,000,000. This investment is significantly 
smaller than the average investment received for programs funded under the Department 
of Defense Acquisition System, which depending on the type of system can be on the 
order of 10’s or 100’s of millions. By limiting the individual program funds, the CTTO 
is able to reduce the amount of risk they bear with any one program. In addition, the 
level of control they exercise over each program directly increases the probability of 
successful development and transference of commercial technologies. Lastly, through its 
policy of investing in mature technologies that can be transitioned to enhance Naval 
capabilities or reduce costs, the CTTO has been able to successfully minimize risk as 
well as cost and schedule impacts. 

2. In-Q-Tel 

In-Q-Tel is the product of the government’s initial attempt at establishing a non¬ 
profit Venture Capital firm. The firm is strategically focused on the funding, 
identification, and transference of commercial technologies to ensure the technological 
superiority of the Central Intelligence Agency (CIA). The impetus for In-Q-Tel’s 
formation was the 1998 Strategic Direction Initiative launched by the CIA’s director, 
George Tenet. The Initiative declared that a new entity was required to speed the 
insertion of mature technologies and to support the rapid development of mission critical 
applications for the CIA and Intelligence Community (IC). 25 With the creation of In-Q- 
Tel the CIA had officially recognized that they could no longer rely solely on its 
traditional contractor base and government laboratories for cutting edge technologies. 26 


25 “Accelerating the Acquisition and Implementation of New Technologies for Intelligence: The 
Report of the Independent Panel on the Central Intelligence Agency In-Q-Tel Venture,” Business 
Executives for National Security (2001), 16 May 2007 http://www.bens.org . 

26 Dupont, Daniel G. “The Company's Company: Venture Capitalism Becomes a New Mission for the 
Nation's Spymasters.” Scientific American (August 2001). 
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a. Objectives and Investment Focus 

Although much of In-Q-Tel is motivated by the practices of successful VC 
firms, In-Q-Tel differs in its objectives and investment focus. Typical VC firms invest in 
technologies across various industry sectors with the goal of generating a significant 
return for their limited partners. Meanwhile, In-Q-Tel is solely concerned with funding 
the development of technologically superior products within the Information Technology 
sector that address its customers’ needs. This includes the following four areas of focus: 

• Information Security: hardening, and intrusion detection, monitoring and 
profiling of information use and misuse, and network and data protection. 

• Use of the Internet: secure receipt of information, non-observable surfing, 
authentication, content verification, and hacker resistance. 

• Distributed Architectures: methods to interface with custom or legacy 
systems, mechanisms to allow dissimilar applications to interact, 
automatic handling of archived data, and connectivity across a wide range 
of environments. 

• Knowledge Generation: geospatial and multimedia data fusion or 
integration, and computer forensics. 27 

In-Q-TeTs success is measured by its ability to “deliver value to the 
Agency through successful deployment of high impact, innovative technologies, build 
strong portfolio companies that will continue to deliver, support, and innovate 
technologies for In-Q-Tel's customers, and create financial returns to fund further 
investments into new technologies.” 28 Equity investments are the vehicle through which 
In-Q-Tel can participate in strategically developing commercial technologies. The 
typical range of investment is between $1,000,000 and $3,000,000. In addition to 
tailoring commercial technologies, these equity investments are made with the goal of a 
future liquidity event for the portfolio company. Examples of liquidity events include 
acquisitions, mergers, 


27 Rick E. Yannuzzi, “In-Q-Tel: A New Partnership between the CIA and the Private Sector.” Defense 
Intelligence Journal (Winter 2000). 

28 “In-Q-Tel - About Us,” In-Q-Tel , 2003, Central Intelligence Agency, 12 May 2007 http://www.in- 
q-tel.org . 
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Initial Public Offering (IPO), etc. These liquidity events provide the means through 
which profits can be reinvested into other promising technologies, thereby creating a self- 
sustainable firm that is no longer dependent upon funding from the CIA. 

b. In-Q-Tel Interface Center (QIC) 

The In-Q-Tel Interface Center arose in response to the many concerns 
raised about security clearances that were often necessary to interact with the CIA. 
Previously, these security concerns had represented a significant hurdle, preventing the 
CIA from detailing its needs to outside suppliers. The QIC was formed to serve as a 
conduit, passing information between the CIA and In-Q-Tel and providing guidance on 
CIA needs and candidate technologies. 29 CIA officers armed with the expertise required 
to translate specific CIA and IC needs into product evaluations and descriptions staff the 
QIC. When In-Q-Tel makes an investment in a company, the QIC is responsible for 
evaluating the technology and strategically placing/developing it for use within one of the 
CIA’s many departments or, if possible, within other government programs. According 
to an independent report by the Business Executives for National Security, the 
implementation of the QIC has been directly responsible for the success of In-Q-Tel. 30 

In-Q-Tel has been able to draw upon the unique experiences of venture 
capitalists, technology executives, and intelligence officers. To date, In-Q-Tel has 
successfully delivered more than 140 technology solutions, engaged more than 90 
companies, most of which had never previously contracted with the government, and 
leveraged more than $1B in private sector funds. 31 These numbers are incontrovertible 
evidence that In-Q-Tel has proven to be a successful application of VC practices within 
the government. 

29 Daniel G. Dupont, “The Company's Company: Venture Capitalism Becomes a New Mission for the 
Nation's Spymasters.” Scientific American (August 2001). 

30 “Accelerating the Acquisition and Implementation of New Technologies for Intelligence: The 
Report of the Independent Panel on the Central Intelligence Agency In-Q-Tel Venture,” Business 
Executives for National Security (2001), 16 May 2007 http://www.bens.org . 

31 “In-Q-Tel - About Us,” In-Q-Tel , 2003, Central Intelligence Agency, 12 May 2007 http://www.in- 
q-tel.org . 
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3. 


OnPoint Technologies 


Given the successful transitions of technology to the CIA and other members of 
the Intelligence Community via In-Q-Tel, the United States Army utilized funds to 
establish its own Army Venture Capital Fund, OnPoint Technologies. OnPoint was 
founded in 2003 as a means of strengthening collaborative ties with young, small, 
entrepreneurial companies that take risks and push innovation. 32 More specifically, 
OnPoint was created to facilitate the rapid development and insertion of promising 
commercial technologies to the Army. 

OnPoint Technologies is a strategic investor motivated by the needs of the Army. 
The Army’s success is directly related to the battlefield effectiveness of mobile soldiers. 
According to the Army, the future mobile soldier “will carry a power/energy source 
dense enough to operate all or most of the power/energy consuming devices needed to 
carry out their missions...operate for days, or even weeks, without re-supply...campaign 
in extreme environments and operate independently, so all of the soldier's equipment, 
including the power/energy sources, must be man-portable over extended distances and 
across extreme terrain.” 33 Armed with these guidelines for what will be expected of 
combat soldiers in the near future, OnPoint focuses on mobile power and energy enabling 
technologies, such as: 

• Generation - fuel cells and microturbines 

• Storage - batteries and capacitors 

• Management - semiconductors and software 

• Controls - control circuits and voltage sensors 

• Distribution - conducting polymers and superconductors 

• Usage - low power logic and components 34 


32 “About OnPoint Technologies,” OnPoint Technologies , 2003, United States Army, 18 May 2007 
http://www.onpoint.us . 

33 Ibid. 

34 Ibid. 
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Similar to In-Q-Tel, OnPoint Technologies has modeled itself as a non-profit 
Venture Capital firm. OnPoint acquires stakes in companies with promising technologies 
through equity investments. These investments range between $500,000 and $2,000,000. 
Each investment is expected to yield a technology that can be tailored to enhance 
capabilities of soldiers in the battlefield. Though the primary criterion for investing in a 
portfolio company is whether the technology can be successfully transitioned to the 
Army, each investment is also evaluated on its potential to reach a liquidity event and 
thus generate a positive return on investment. Attractive companies meet both of these 
criteria. To date, OnPoint has made investments in seven promising power and energy 
technology companies, with one successful technology transition to the Army - a cutting- 
edge power indicator. Placing this power indicator on the BA5590A battery, the most 
widely used battery in the Army, is expected to save millions of dollars for the Army. 
Despite the fact that OnPoint is a relatively new player to the DoD Venture Capital 
forum, it is following in the footsteps of its parent model, In-Q-Tel. It is likely that 
OnPoint will attain a similar level of success with respect to its current and future 
investments. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. ADVANTAGES OF GOVERNMENT VENTURE CAPITAL PRACTICES 

The Department of Defense continues to face numerous challenges in working 
cooperatively with the smaller companies that rapidly develop innovative products. From 
the small business’ perspective, conducting business with the Department of Defense 
often means subjecting themselves to a multitude of regulations and security constraints 
they would not face in the commercial marketplace. In addition to the DoD’s long and 
unpredictable budget cycle, there is often little or no incentive to market their products or 
services to the military, given it may take two to three years to potentially sell it to the 
DoD compared to an 18-month timeline in the commercial sector. 35 Therefore, the 
Department of Defense has explored venture capital models to cultivate an environment 
in which commercially viable technologies can be rapidly transferred to enhance 
warfighter capabilities. 

There have been numerous advantages to the Department of Defense’s adoption 
of the tailored VC models, In-Q-Tel and OnPoint Technologies. In-Q-Tel’s strategic 
focus on the CIA and IC has enabled them to successfully focus its investments within 
the Information Technology sector. Meanwhile, OnPoint Technologies has narrowed its 
investment focus to the energy and power sectors to improve the mobility of the Army 
soldier. Through the practices of incremental funding and deal syndication, both In-Q- 
Tel and OnPoint are granted access to leading-edge commercial technologies, increased 
visibility into portfolio company operations, increased risk management, and the ability 
to operate outside of the bureaucratic constraints of the Department of Defense 
Acquisition System. 


35 “Navy Needs to Define Path for IT Industries to Market their Wares.” C4I News . (August 2005). 
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1 . 


Access to Leading-Edge Commercial Technologies 


The bulk of innovation has been occurring in young commercial companies that 
have little experience, or visibility, with the government. 36 Equity investments between 
$500,000 and $3,000,000 enable the DoD VC firms to directly fund the research and 
development of multiple mission critical technologies at any given time. “The 
conventional contractor deal might be strategic but it doesn’t give you the same insight 
into the company or ability to influence the company as an equity role does.” 37 In 
addition to funding promising technologies, each is able to increase its exposure to 
commercial companies, thereby granting access to a largely untapped market. Since 
more than 75% of the companies that In-Q-Tel works with have never done business with 
the government, it is evident that this approach provides an attractive alternative means 
for new companies to deliver technology solutions to the Department of Defense. 38 

2. Visibility into Portfolio Company Operations 

Achieving a sufficient level of visibility into contracted company operations has 
been a persistent challenge for almost all DoD Acquisition Programs. Through the 
implementation of venture capital practices, In-Q-Tel and OnPoint have been able to 
increase the level of visibility they have into commercial companies. The amount of due 
diligence they conduct is comparable to other DoD programs. However, due to uniquely 
aligned interests between a candidate company and its potential investors, there is 
significantly less room for bureaucratic maneuvering when it comes to supplying the 
necessary documentation that confirms certain investment criterion have been met. Once 
In-Q-Tel or OnPoint have invested, it is not uncommon for their partners to be offered a 
seat on the company’s board of directors. Acceptance of board membership grants them 
additional visibility into the portfolio firm’s operations, directly increasing the level of 

36 “In-Q-Tel - About Us,” In-Q-Tel , 2003, Central Intelligence Agency, 12 May 2007 http://www.in- 
q-tel.org . 

37 Gary Aden, “For Tech’s Sake: Can In-Q-Tel’s investment model translate into other agencies?” 
Washington Technology (22 April 2004). 20 May 2007 http://www.washingtontechnology.com . 

38 “In-Q-Tel - About Us,” In-Q-Tel , 2003, Central Intelligence Agency, 12 May 2007 http://www.in- 
q-tel.org . 
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control they can exercise throughout the technology’s development. Therefore, through 
board membership and significant due diligence In-Q-Tel and OnPoint have been able to 
increase their visibility into portfolio company operations, contributing to the increased 
number of successful technology acquisitions. 

3. Risk Management 

Risk management is a topic every acquisition professional is intimately familiar 
with, yet, still acquisition programs continue to fail throughout the DoD due to poor 
decision-making. Two methods of managing risk that have proved successful for the 
Department of Defense VC firms has been the adoption of incremental funding and deal 
syndication. The practice of incremental funding involves spreading out the funding of a 
technology over periods of time, or rounds. Funds are raised at the onset of each round, 
which is determined by the successful completion of pre-specified milestones. Deal 
syndication, a standard practice throughout the VC industry, involves generating enough 
interest in a portfolio firm's technology to recruit additional venture investors. As more 
investors contribute funds to the commercial technology, the level of risk each investor is 
exposed to on any one investment is correspondingly reduced. Since In-Q-Tel and 
OnPoint syndicate all investments, their total exposure to any individual portfolio 
company is limited through this practice of risk diversification. Another advantage to 
syndicating deals is the opportunity it presents to leverage the capital of additional 
investors, allowing In-Q-Tel and OnPoint to invest in other promising technologies. 

As mentioned previously in this paper, the Department of Defense Acquisition 
System has not been able to keep pace with the development of technology in 
commercial industry. The bureaucratic constraints of traditional acquisition programs 
have been responsible for unpredictable annual funding and poor risk management, both 
of which result in cost and schedule overruns that directly impede the transition of 
technologies from concept to deployment. Simply put, companies that are not part of the 
traditional contractor base don’t want the headaches associated with government 
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accounting and acquisition regulations. 39 The formation of Venture Capital models has 
provided a successful vehicle for accessing the entrepreneurial community, which is at 
the forefront of developing today’s innovative technologies. In-Q-Tel and OnPoint 
Technologies have proven to be capable methods of rapidly identifying, acquiring, and 
transferring commercial technologies, while minimizing risks and maximizing the 
number of successful acquisitions. However, the incorporation of a full-fledged VC 
model to improve DoD software acquisitions is unnecessary. Certain characteristics of 
an acquisition project lend itself to the incorporation of venture capital practices. The 
development of technology by a young company or startup, increases the overall level of 
risk associated with the program. In addition to technology risk, there is the introduction 
of company risk, the risk of bankruptcy, merger, or acquisition, which could delay or 
even prevent the technology’s development. In addition to elevated risk, if the 
technology developed met the needs of several DoD-affiliated organizations, it would be 
an ideal candidate for the implementation of venture capital practices. The 
implementation of venture capital practices should be used to complement the 
conventional acquisition process, rather than replace it. Therefore, incremental funding 
and deal syndication should be applied to improve the number of successful software 
acquisitions. 

B. RECOMMENDATIONS 

The most common reasons for why software acquisitions typically fail have been 
identified throughout this paper. The Department of Defense Acquisition System is a 
long and tedious process, filled with numerous regulations and restrictions, which serve 
to manage the risks associated with procuring technology. However, year-after-year this 
Acquisition System has resulted in a significant number of software acquisition failures. 
There are many current practices that can be improved or modified based on the analysis 
of comparable commercial practices and the advantages of venture capital investment and 


39 Daniel G. Dupont, “The Company's Company: Venture Capitalism Becomes a New Mission for the 
Nation's Spymasters.” Scientific American (August 2001). 
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management practices, many of which have been discussed in this chapter. The 
following are a few additional recommendations that should be considered to improve the 
regulations governing the acquisition of software systems. 

• More aggressive metrics for vendor selection to measure both current 
ability to perform and past performance. It is necessary to improve the 
level of visibility the Department of Defense has into contracted 
companies; often it is too late before issues are recognized, addressed, and 
resolved. 

• Encourage collaboration among competing vendors on large complex 
software systems. This practice would reduce the overall risk of each party 
involved, by enabling the DoD to award smaller amounts of capital to 
each vendor, thereby forcing them to utilize their individual resources 
efficiently and resulting in improved cost management practices. 

• Push for the incorporation of open source standards into software systems 
to maximize the amount of software reuse within the DoD. Software 
reuse will minimize the cost of procuring future software systems, while 
also reducing some of the risks associated with software development. 

• Software architectures should be open, allowing for the incorporation of 
rapid changes and providing the ability for multiple vendors to contribute 
through the development of critical components and/or customized 
modules for software systems. Through the use of modules, the set of 
requirements can be broken down into subsets of user requirements or 
individual components designed to complete specific tasks. Modules 
increase the level of customization that can be achieved for each user, by 
allowing the same software system to satisfy the needs of multiple users. 
The goal should be to build smaller, simpler, cleaner software components 
that can easily be incorporated into existing open software architectures. 

• Facilitate iterative software development through either incremental or 
evolutionary models. These models are particularly advantageous since 
they yield a working product at the end of each iteration. Thus, in any 
instance where additional funds cannot be appropriated or cost and 
schedule overruns lead to program termination, the program will have 
successfully yielded a working prototype that could be further developed 
by another program office or incorporated into an existing system. These 
models incorporate the newest technology in each successive prototype. 
Additionally, software requirements are continuously improved and 
aligned with user needs. Iterative software development is ideal for 
medium to high-risk projects, where requirements are expected to evolve 
due to a large number of diverse users each with their own set of needs. 

• Function points should be the primary method for detennining software 
development cost. The function point method focuses on five different 


41 



factors that relate to user requirements: inputs, outputs, logic files, 
inquiries, and interfaces. Under this method, a secondary system tracks 
the number of times each one of the function points is executed in the 
software under development and then total cost is calculated by 
multiplying the number of times each function point is 
performed by its respective cost. Thus, the software system’s size is based 
on user-defined functionality and not the number of lines of code required 
to complete the task. 

• Relaxation and/or modification of rotation assignments for DoD software 
acquisition professionals. Program managers should be required to stay 
with programs throughout initial testing to maintain continuity and 
understanding of original requirement nuances. Their unique 
understanding of the software development is crucial to the program’s 
success. 

• Syndication of projects across various program offices and services of the 
DoD. Deal syndication has proven extremely successful throughout the 
VC industry in increasing the successful development of commercial 
technologies. By seeking out interested parties, program risk and funding 
can be split amongst the group, thereby reducing any one program office’s 
or service’s exposure to program risk. Therefore, if any one of the 
invested parties cannot appropriate its portion of the program’s funds, the 
chances of project termination are reduced. 
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